Skip to content

feat: Add Isolate kit - #291

Open
Dhruv2mars wants to merge 76 commits into
Lamatic:mainfrom
Dhruv2mars:feat/isolate
Open

feat: Add Isolate kit#291
Dhruv2mars wants to merge 76 commits into
Lamatic:mainfrom
Dhruv2mars:feat/isolate

Conversation

@Dhruv2mars

@Dhruv2mars Dhruv2mars commented Jul 24, 2026

Copy link
Copy Markdown

Isolate

Turn vague GitHub issues into verified reproduction evidence.

Live demo · Evaluation issue · Kit documentation

Isolate showing a reproduced outcome with two passing candidate runs and a rejected negative control

Problem

Open-source maintainers regularly receive issues that describe a symptom but omit the exact environment and commands needed to reproduce it. An AI agent can propose a plausible explanation, but its confidence is not evidence.

Isolate turns a public GitHub issue into a repeatable, machine-verifiable reproduction report inside a disposable sandbox.

What this kit adds

  • A deployed Lamatic planner flow that interprets normalized issue and repository context
  • A production Next.js reviewer interface with Markdown and JSON report export
  • Disposable Daytona sandboxes with deterministic dependency setup and immutable ref support
  • An authenticated HTTP MCP runtime for saved Lamatic MCP/Tools connections
  • A strict command policy limited to repository-owned package scripts
  • Bounded, redacted stdout/stderr plus exit code and duration evidence
  • Aggregate deadlines, concurrency protection, network isolation, and verified cleanup
  • A controlled public CLI fixture for repeatable evaluation

Trust boundary

The model investigates. The runtime verifies.

Lamatic forms a hypothesis and proposes a candidate probe plus a nearby negative control. It cannot mark its own work reproduced. The runtime derives one exact assertion from the issue and certifies reproduced only when:

  1. candidate run 1 passes;
  2. candidate run 2 passes again; and
  3. the negative control rejects the same assertion.

If that gate does not pass, Isolate returns not_reproduced_under_tested_conditions or blocked.

Architecture

flowchart LR
    A[Public GitHub issue] --> B[Deterministic intake]
    B --> C[Disposable Daytona sandbox]
    C --> D[Lamatic probe planner]
    D --> E[Runtime command policy]
    E --> F[Candidate twice plus control]
    F --> G[Deterministic evidence gate]
    G --> H[Markdown and JSON report]
Loading

Reviewer path

  1. Open the live demo.
  2. Click Use evaluation fixture (or paste another supported public issue), then click Run isolation.
  3. Inspect the hypothesis and the three recorded runs.
  4. Confirm the two candidate assertions pass, the negative control rejects, and sandbox deletion is reported.
  5. Download either portable report format.

The fixture describes the symptom and observed output without supplying the reproduction command. Isolate must inspect the repository and discover the correct repository-owned CLI invocation.

Verification

  • Live Lamatic planner flow deployed
  • Live Vercel application returns HTTP 200
  • Authenticated MCP discovery and tool execution verified
  • Real Daytona sandbox creation, public clone, probe execution, and deletion verified
  • Evaluation issue reproduced repeatedly with two candidate passes and a rejecting control
  • 94 automated tests passing (263 assertions)
  • TypeScript check passing
  • Next.js production build passing
  • AgentKit structural validation passing
  • No credentials committed

Scope

The initial release supports public Node.js, TypeScript, Bun, and terminal/CLI issues. Terminal-output certification uses an exact issue-derived stdout/stderr signature. For unsaved-exit TUI issues, the runtime drives a real PTY, verifies unchanged file state twice, and rejects the claim after a save control. Other issue classes remain blocked without a runtime-owned evidence adapter. Isolate does not edit repository source files, generate fixes, mount repository credentials, push branches, open pull requests, or publish packages.

Challenge checklist

  • Unique contribution under kits/isolate
  • Exported and deployed Lamatic flow with externalized references
  • Runnable kit application and one-click Vercel configuration
  • Public live demo and reviewer fixture
  • agentkit-challenge label applied
  • Required AgentKit files and links present

Status

Ready for final review. Implementation, submission materials, live demo, and reviewer walkthrough are complete.

Files Added

Configuration and documentation:

  • kits/isolate/.env.example
  • kits/isolate/.gitignore
  • kits/isolate/README.md
  • kits/isolate/agent.md
  • kits/isolate/constitutions/default.md ("Isolate Constitution")
  • kits/isolate/lamatic.config.ts
  • kits/isolate/model-configs/isolate-reproduction-model.ts
  • kits/isolate/prompts/isolate-reproduction-system.md
  • kits/isolate/prompts/isolate-reproduction-user.md

Lamatic flow:

  • kits/isolate/flows/isolate-reproduction.ts (Lamatic flow definition)

Next.js application core:

  • kits/isolate/apps/.env.example
  • kits/isolate/apps/.gitignore
  • kits/isolate/apps/DESIGN.md
  • kits/isolate/apps/PRODUCT.md
  • kits/isolate/apps/styles.d.ts
  • kits/isolate/apps/next-env.d.ts
  • kits/isolate/apps/next.config.ts
  • kits/isolate/apps/package.json
  • kits/isolate/apps/tsconfig.json
  • kits/isolate/apps/app/globals.css
  • kits/isolate/apps/app/layout.tsx
  • kits/isolate/apps/app/page.tsx

API routes:

  • kits/isolate/apps/app/api/investigate/route.ts
  • kits/isolate/apps/app/api/mcp/route.ts

Frontend components:

  • kits/isolate/apps/components/investigation-workbench.tsx

Runtime and service libraries:

  • kits/isolate/apps/lib/concurrency.ts
  • kits/isolate/apps/lib/deadline.ts
  • kits/isolate/apps/lib/http-errors.ts
  • kits/isolate/apps/lib/investigate.ts
  • kits/isolate/apps/lib/investigation-request.ts
  • kits/isolate/apps/lib/lamatic-planner.ts
  • kits/isolate/apps/lib/runtime/certification.ts
  • kits/isolate/apps/lib/runtime/claim.ts
  • kits/isolate/apps/lib/runtime/daytona.ts
  • kits/isolate/apps/lib/runtime/evidence.ts
  • kits/isolate/apps/lib/runtime/github.ts
  • kits/isolate/apps/lib/runtime/investigation-report.ts
  • kits/isolate/apps/lib/runtime/mcp.ts
  • kits/isolate/apps/lib/runtime/plan.ts
  • kits/isolate/apps/lib/runtime/policy.ts
  • kits/isolate/apps/lib/runtime/probe.ts

Test suite:

  • kits/isolate/apps/tests/certification.test.ts
  • kits/isolate/apps/tests/claim.test.ts
  • kits/isolate/apps/tests/concurrency.test.ts
  • kits/isolate/apps/tests/daytona.test.ts
  • kits/isolate/apps/tests/deadline.test.ts
  • kits/isolate/apps/tests/evidence.test.ts
  • kits/isolate/apps/tests/github.test.ts
  • kits/isolate/apps/tests/http-errors.test.ts
  • kits/isolate/apps/tests/investigate.test.ts
  • kits/isolate/apps/tests/investigation-report.test.ts
  • kits/isolate/apps/tests/investigation-request.test.ts
  • kits/isolate/apps/tests/lamatic-planner.test.ts
  • kits/isolate/apps/tests/mcp.test.ts
  • kits/isolate/apps/tests/plan.test.ts
  • kits/isolate/apps/tests/probe.test.ts

Assets:

  • kits/isolate/assets/isolate-evidence.jpg

Lamatic Flow (kits/isolate/flows/isolate-reproduction.ts)

Node types introduced:

  • triggerNode_1 (type: triggerNode, "API Request"): Receives advance_schema containing issue, repositoryContext, ref, and policyFeedback fields from incoming API request.
  • LLMNode_887 (type: dynamicNode, "Generate Text"): Executes system and user prompts (@prompts/isolate-reproduction-system.md, @prompts/isolate-reproduction-user.md) using generative model configuration from @model-configs/isolate-reproduction-model.ts (Gemini 3.1 Flash Lite).
  • responseNode_triggerNode_1 (type: responseNode, "API Response"): Maps LLM output to JSON response by embedding LLMNode_887.output.generatedResponse into a plan field with application/json content type.

Data flow:

  • triggerNode_1 forwards API request to LLMNode_887 via default edge.
  • LLMNode_887 processes issue context and produces reproduction plan via LLM inference.
  • responseNode_triggerNode_1 receives LLM output and returns structured plan response via default edge.
  • Response edge directly connects triggerNode_1 to responseNode_triggerNode_1 for response routing.

High-level behavior:
The flow receives a GitHub issue, repository snapshot, optional commit reference, and policy feedback. The LLM node interprets the issue as untrusted input and produces a structured reproduction plan that specifies either terminal-based candidate/control commands or TUI-based unsaved-exit probe instructions. The runtime then executes this plan in an isolated Daytona sandbox, certifies candidate and control probe evidence against the issue's observed assertion, and reports pass/fail outcome based on the evidence gate criteria (two candidate passes and negative control rejection).

@coderabbitai

coderabbitai Bot commented Jul 24, 2026

Copy link
Copy Markdown
Contributor

Review Change Stack

Walkthrough

The PR adds the Isolate reproduction kit. It includes Lamatic planning, Daytona sandbox certification, MCP and HTTP interfaces, a Next.js workbench, safety and evidence contracts, deployment configuration, documentation, and Bun tests.

Changes

Isolate reproduction workflow

Layer / File(s) Summary
Contracts and planning flow
kits/isolate/README.md, kits/isolate/agent.md, kits/isolate/constitutions/*, kits/isolate/flows/*, kits/isolate/prompts/*, kits/isolate/model-configs/*, kits/isolate/lamatic.config.ts, kits/isolate/.env.example, kits/isolate/apps/.env.example
Defines the Isolate workflow, safety rules, Lamatic planner flow, model configuration, deployment metadata, documentation, and environment templates.
Runtime contracts and certification
kits/isolate/apps/lib/runtime/{claim,plan,policy,probe,certification,evidence,investigation-report}.ts, kits/isolate/apps/lib/deadline.ts, kits/isolate/apps/lib/investigation-request.ts
Adds schemas and logic for issue evidence, reproduction plans, command validation, probe evaluation, deadlines, certification gates, reports, and request shaping.
Investigation orchestration and providers
kits/isolate/apps/lib/investigate.ts, kits/isolate/apps/lib/runtime/{daytona,github,mcp}.ts, kits/isolate/apps/lib/lamatic-planner.ts, kits/isolate/apps/app/api/*
Connects GitHub issue intake, Lamatic planning and reporting, Daytona sandbox execution, MCP tools, HTTP routes, concurrency limits, certification, and cleanup.
Runtime integration validation
kits/isolate/apps/tests/*
Adds Bun tests covering runtime contracts, Daytona lifecycle and security behavior, investigation orchestration, Lamatic integration, MCP handling, HTTP errors, and concurrency.
Web application and project setup
kits/isolate/apps/app/*, kits/isolate/apps/components/*, kits/isolate/apps/package.json, kits/isolate/apps/tsconfig.json, kits/isolate/apps/next*.{ts,d.ts}, kits/isolate/apps/.gitignore
Adds the Next.js application shell, responsive styling, investigation form and results UI, export actions, API wiring, project scripts, dependencies, and TypeScript configuration.

Suggested reviewers: amanintech, d-pamneja

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 1.56% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
Title check ✅ Passed The title clearly identifies the primary change: adding the Isolate kit.
Description check ✅ Passed The description is detailed and covers the kit purpose, architecture, scope, validation results, deployment status, and security constraints.
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@github-actions

github-actions Bot commented Jul 24, 2026

Copy link
Copy Markdown
Contributor

:robot_face: AgentKit Structural Validation

New Contributions Detected

  • Kit: kits/isolate

Check Results

Check Status
No edits to existing kits ✅ Pass
Required root files present ✅ Pass
Flow .ts files present ✅ Pass
lamatic.config.ts valid ✅ Pass
No changes outside kits/ ✅ Pass

🎉 All checks passed! This contribution follows the AgentKit structure.

@github-actions

Copy link
Copy Markdown
Contributor

Failure recorded at 2026-07-24T10:48:07Z UTC. If this PR is not fixed within 4 weeks it will be automatically closed.

@Dhruv2mars
Dhruv2mars marked this pull request as ready for review July 30, 2026 16:54
@coderabbitai
coderabbitai Bot requested a review from d-pamneja July 30, 2026 16:56

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 33

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@kits/isolate/agent.md`:
- Around line 19-22: Update the supported-scope statement in the capability
document to include Bun alongside Node.js and TypeScript, while preserving the
existing repository and modification restrictions.

In `@kits/isolate/apps/lib/concurrency.ts`:
- Around line 1-14: Extend acquireInvestigationSlot beyond the process-local
activeInvestigations counter by backing it with a shared limiter such as
Redis/Upstash or a provider-side quota, while retaining the local gate as a
fallback when the shared limiter is unavailable. Document in the kit README that
the local maximumConcurrentInvestigations limit is instance-scoped and does not
cap concurrency across autoscaled Vercel instances.

In `@kits/isolate/apps/lib/investigate.ts`:
- Around line 118-140: The model-call retry logic retries every failure
immediately and unconditionally. In
kits/isolate/apps/lib/investigate.ts:118-140, extract a shared retry helper that
classifies errors, stops immediately for deterministic failures, applies backoff
for transient or rate-limit failures, and respects the remaining deadline;
update requestPlan to use it. In kits/isolate/apps/lib/investigate.ts:216-225,
replace the unconditional requestReport retry with the same helper, preserving
the existing requestReport behavior while applying identical error handling.
- Around line 226-240: Restrict issueRequiresInteractiveEvidence in the
investigation flow to explicit interaction signals such as keypress, keyboard,
TUI, or pressing a control, removing generic matches for save, saving, editor,
click, type, and typing. Keep the existing report.limitations evidence-gap check
and inconclusive outcome handling unchanged.
- Around line 18-29: Cap the repository snapshot output before constructing the
planner context, using the same 45,000-character bound as the evidence/report
payload. Update the relevant investigate flow to derive and reuse
repositoryContext from the bounded snapshot, replacing direct
snapshot.observation.stdout usage at the claim-derivation, planner, and
subsequent context call sites.
- Around line 327-330: Update the cleanup flow surrounding runtime.delete so a
deletion failure preserves the original investigation error when the main
operation already failed, while still failing closed when deletion cannot be
confirmed. Restructure the success paths to delete before returning, or
otherwise combine both failures without replacing the original cause, using the
existing investigate flow and runtime.delete call.

In `@kits/isolate/apps/lib/lamatic-planner.ts`:
- Around line 72-81: Update the response-handling flow around response.json() to
parse defensively, catching malformed or empty upstream bodies instead of
allowing a raw parsing error to escape. In the planner request logic, ensure
response.ok and the parsed error message determine the thrown error, falling
back to “Lamatic could not produce a probe plan.” when the body cannot be parsed
or has no usable message; preserve successful execution handling via
executeWorkflow.
- Around line 55-71: Update executeWorkflow to invoke the sanctioned lamatic SDK
flow-call API instead of manually constructing the GraphQL request,
authentication, and project headers. Preserve the existing input,
policyFeedback, endpoint, and per-request timeout behavior; if the SDK cannot
accept the timeout signal, document that limitation with a brief comment and
dependency rationale.

In `@kits/isolate/apps/lib/runtime/claim.ts`:
- Around line 50-71: Restrict the tui_unsaved_exit classification in the claim
logic so repository-wide context cannot independently satisfy the key signals.
Require namesQuitShortcut to match the issue text, while allowing
namesTerminalSurface to use contextualText only as corroboration; preserve the
existing namesExit and namesUnsavedState checks.

In `@kits/isolate/apps/lib/runtime/daytona.ts`:
- Around line 668-676: Update the cleanup in the installation flow’s finally
block to catch and log failures from updateNetworkPolicy when re-blocking the
network, while preserving the original installation error. Track whether the
main body succeeded, and only rethrow the cleanup failure when no body error
occurred, matching the behavior used by runProbe.
- Around line 394-417: Update runProbe in
kits/isolate/apps/lib/runtime/daytona.ts at lines 394-417 to capture the body
error, log and swallow cleanup failures, rethrowing the body error first and
surfacing cleanup failure only when the body succeeded. At
kits/isolate/apps/lib/runtime/daytona.ts lines 298-324, attach logging to
deleteSandbox cleanup so “Repository cloning failed.” remains the propagated
error; at lines 668-676, log and swallow re-block updateNetworkPolicy failures
so “Deterministic dependency installation failed.” survives. In
kits/isolate/apps/lib/runtime/mcp.ts lines 198-219, log and swallow
runtime.delete failures in the inner finally so completed verdicts are returned.
- Around line 550-565: Make the TUI readiness polling around the
alternate-screen marker an explicit success condition before sending any
keystrokes. Track whether output contains "\u001b[?1049h"; if the 40-attempt
loop completes without detecting it, fail or throw through the existing
certification error path and do not execute the subsequent ESC, edit, save, or
quit sends. Preserve the current interaction sequence only when readiness is
confirmed.
- Around line 298-324: Update the catch block in the sandbox setup flow so the
original cloning or network-policy error remains the thrown error when
deleteSandbox cleanup fails. Catch cleanup failures from deleteSandbox, preserve
the original error, and attach the cleanup failure as its cause or otherwise
retain it without replacing the primary error.
- Around line 378-386: Update the collection flow around execute and the
observation parse to validate collected.exitCode before calling JSON.parse,
matching the guarded behavior in runTuiUnsavedExitProbe. On nonzero exit, fail
closed with a diagnosable error identifying the collector probe and relevant
stderr; only parse and validate the JSON result when execution succeeds.

In `@kits/isolate/apps/lib/runtime/github.ts`:
- Around line 49-58: Update the request options in the issue-reading method
containing the GitHub API call so the caller-provided options.signal is composed
with a separate 10-second AbortSignal.timeout(10_000), rather than selected via
nullish coalescing. Preserve both cancellation sources: caller deadlines must
still abort the request, and the GitHub fetch must always enforce its own
10-second timeout.

In `@kits/isolate/apps/lib/runtime/investigation-report.ts`:
- Around line 3-4: Update investigationReportSchema so model output cannot
define the final reproduction classification: remove the outcome field and
inject the deterministic certification result when constructing the report, or
replace it with the canonical runtime-owned certification enum. Ensure the
report uses the certified result rather than the model-specific
likely_reproduced value.

In `@kits/isolate/apps/lib/runtime/mcp.ts`:
- Around line 232-251: Harden the authorization gate in the MCP request handler
by parsing the Authorization header into scheme and token, accepting the scheme
case-insensitively, and comparing the token to secret with
crypto.timingSafeEqual only after confirming equal-length buffers. Preserve the
existing 401 response for missing, malformed, or mismatched credentials.

In `@kits/isolate/apps/lib/runtime/policy.ts`:
- Line 8: Update the environment-dump regex in the policy tripwire to match env,
printenv, or set only when they occur in executable command position, following
the existing sudo rule’s anchoring approach. Preserve rejection of direct
environment-dump commands while allowing those words in script names, arguments,
paths, and subcommands.

In `@kits/isolate/apps/package.json`:
- Around line 13-14: Update the `@modelcontextprotocol/server` dependency in the
package manifest to use an exact pinned beta version instead of the caret range,
and apply the same deterministic versioning to the matching
`@modelcontextprotocol/client` dependency. Preserve compatibility between the two
MCP packages and ensure neither can resolve to an unintended pre-release update.
- Line 26: Update compilerOptions.types in kits/isolate/apps/tsconfig.json to
include node, react, and react-dom alongside bun so the app can resolve the
expected type packages under TypeScript 6.0.

In `@kits/isolate/apps/tests/claim.test.ts`:
- Around line 9-36: Add focused assertions in the “issue evidence contract”
tests that exercise extractIssueEvidenceAssertion with an Observed stderr
backtick value and with an unquoted plain value, verifying the expected
stderr_contains and stdout_contains results respectively. Keep the existing
rejection cases unchanged.

In `@kits/isolate/apps/tests/concurrency.test.ts`:
- Around line 11-21: Extend the test named “bounds expensive work per
application instance” to invoke the same release closure twice and verify that
the second invocation does not free another slot. Confirm the slot remains
unavailable until the other acquired slot is released, then preserve the
existing assertion that a slot becomes available after a legitimate release.

In `@kits/isolate/apps/tests/evidence.test.ts`:
- Around line 33-85: Add tests in the certifyEvidence suite covering
candidateRuns with one run and with three runs, asserting neither is certified
as reproduced and that the gate repeatCount reflects the supplied arity. Keep
the existing two-run certification test unchanged so the exact repeatCount === 2
boundary is pinned.

In `@kits/isolate/apps/tests/investigate.test.ts`:
- Line 460: Rename the tests at the indicated locations to clearly state that
the outcome is downgraded to “inconclusive” when the limited exploratory probe
is not reproduced and when the static launch likely indicates an interactive
reproduction. Keep the test behavior and assertions unchanged.
- Around line 174-195: Add a test case for an explicit issue ref through
investigateIssue, configuring the issue input with a non-default ref and
asserting the ref is included in the runtime.create inputs and passed to the
planner. Preserve the existing default-branch test and verify the investigation
still uses the requested ref.

In `@kits/isolate/apps/tests/investigation-report.test.ts`:
- Around line 5-21: Extend the parseInvestigationReport tests with rejection
cases for malformed JSON, an unsupported outcome value, and a report missing
markdown. Assert each input throws or is otherwise rejected, while preserving
the existing valid bounded-report test.

In `@kits/isolate/apps/tests/investigation-request.test.ts`:
- Around line 20-36: Add tests covering the rejection behavior of
investigationRequest for a pull request URL, a non-GitHub issues URL, and a
malformed URL. Assert the function’s actual failure contract—return null or an
unsuccessful safeParse result rather than throwing, as implemented—while
preserving the existing valid-URL cases.

In `@kits/isolate/apps/tests/lamatic-planner.test.ts`:
- Around line 55-76: Add failure-path tests for requestLamaticReport covering
non-ok HTTP responses, status values other than “success,” and passthrough of
errors[0].message. Add assertions that the request sends the expected
Authorization and x-project-id headers, preserving the contract used by
investigate.ts retries.

In `@kits/isolate/apps/tests/mcp.test.ts`:
- Around line 41-57: Expand the authorization tests around handleMcp to cover
both a presented but incorrect bearer token and a defined authorization header
when the configured secret is undefined. Assert that each request is rejected
with HTTP 401, preserving the existing missing-header case and verifying exact
token matching plus fail-closed behavior.

In `@kits/isolate/apps/tests/plan.test.ts`:
- Around line 63-178: Refactor the single test around assertCertificationCommand
into table-driven rejection cases executed with test.each, giving each command
vector a descriptive name and expected output where needed. Keep the existing
accepted-command assertions separate, and preserve every current rejection
vector so failures are reported independently with the specific escape hatch
identified.

In `@kits/isolate/apps/tsconfig.json`:
- Around line 2-15: Enable noUncheckedIndexedAccess in the compilerOptions of
tsconfig.json, then resolve the resulting type errors across the affected
token-parsing code, including the matches[0] destructuring in claim.ts, by
explicitly handling potentially undefined indexed values while preserving
existing command-safety behavior.
- Around line 16-18: Update the compilerOptions.types configuration in
tsconfig.json so the Next app does not globally include Bun types; remove the
explicit "bun" entry or replace it with the appropriate Next-specific types
configuration while preserving the rest of the TypeScript settings.

In `@kits/isolate/lamatic.config.ts`:
- Line 24: Update the docs entry in the Isolate kit configuration to remove the
supervisor-agent URL; either point it to an existing Isolate-specific landing or
README URL, or remove the docs property until such documentation exists.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Repository UI (base), Organization UI (inherited)

Review profile: ASSERTIVE

Plan: Pro Plus

Run ID: 960caa46-f778-499f-a2d4-5ce4be8e25ae

📥 Commits

Reviewing files that changed from the base of the PR and between 2f0b57e and f36ec09.

⛔ Files ignored due to path filters (2)
  • kits/isolate/apps/bun.lock is excluded by !**/*.lock
  • kits/isolate/assets/isolate-evidence.jpg is excluded by !**/*.jpg
📒 Files selected for processing (55)
  • kits/isolate/.env.example
  • kits/isolate/.gitignore
  • kits/isolate/README.md
  • kits/isolate/agent.md
  • kits/isolate/apps/.env.example
  • kits/isolate/apps/.gitignore
  • kits/isolate/apps/DESIGN.md
  • kits/isolate/apps/PRODUCT.md
  • kits/isolate/apps/app/api/investigate/route.ts
  • kits/isolate/apps/app/api/mcp/route.ts
  • kits/isolate/apps/app/globals.css
  • kits/isolate/apps/app/layout.tsx
  • kits/isolate/apps/app/page.tsx
  • kits/isolate/apps/components/investigation-workbench.tsx
  • kits/isolate/apps/lib/concurrency.ts
  • kits/isolate/apps/lib/deadline.ts
  • kits/isolate/apps/lib/http-errors.ts
  • kits/isolate/apps/lib/investigate.ts
  • kits/isolate/apps/lib/investigation-request.ts
  • kits/isolate/apps/lib/lamatic-planner.ts
  • kits/isolate/apps/lib/runtime/certification.ts
  • kits/isolate/apps/lib/runtime/claim.ts
  • kits/isolate/apps/lib/runtime/daytona.ts
  • kits/isolate/apps/lib/runtime/evidence.ts
  • kits/isolate/apps/lib/runtime/github.ts
  • kits/isolate/apps/lib/runtime/investigation-report.ts
  • kits/isolate/apps/lib/runtime/mcp.ts
  • kits/isolate/apps/lib/runtime/plan.ts
  • kits/isolate/apps/lib/runtime/policy.ts
  • kits/isolate/apps/lib/runtime/probe.ts
  • kits/isolate/apps/next-env.d.ts
  • kits/isolate/apps/next.config.ts
  • kits/isolate/apps/package.json
  • kits/isolate/apps/tests/certification.test.ts
  • kits/isolate/apps/tests/claim.test.ts
  • kits/isolate/apps/tests/concurrency.test.ts
  • kits/isolate/apps/tests/daytona.test.ts
  • kits/isolate/apps/tests/deadline.test.ts
  • kits/isolate/apps/tests/evidence.test.ts
  • kits/isolate/apps/tests/github.test.ts
  • kits/isolate/apps/tests/http-errors.test.ts
  • kits/isolate/apps/tests/investigate.test.ts
  • kits/isolate/apps/tests/investigation-report.test.ts
  • kits/isolate/apps/tests/investigation-request.test.ts
  • kits/isolate/apps/tests/lamatic-planner.test.ts
  • kits/isolate/apps/tests/mcp.test.ts
  • kits/isolate/apps/tests/plan.test.ts
  • kits/isolate/apps/tests/probe.test.ts
  • kits/isolate/apps/tsconfig.json
  • kits/isolate/constitutions/default.md
  • kits/isolate/flows/isolate-reproduction.ts
  • kits/isolate/lamatic.config.ts
  • kits/isolate/model-configs/isolate-reproduction-model.ts
  • kits/isolate/prompts/isolate-reproduction-system.md
  • kits/isolate/prompts/isolate-reproduction-user.md

Comment thread kits/isolate/agent.md Outdated
Comment thread kits/isolate/apps/lib/concurrency.ts
Comment thread kits/isolate/apps/lib/investigate.ts
Comment thread kits/isolate/apps/lib/investigate.ts
Comment thread kits/isolate/apps/lib/investigate.ts
Comment thread kits/isolate/apps/tests/mcp.test.ts
Comment thread kits/isolate/apps/tests/plan.test.ts
Comment thread kits/isolate/apps/tsconfig.json
Comment thread kits/isolate/apps/tsconfig.json
Comment thread kits/isolate/lamatic.config.ts Outdated
@Dhruv2mars

Copy link
Copy Markdown
Author

@coderabbitai review

@coderabbitai

coderabbitai Bot commented Jul 30, 2026

Copy link
Copy Markdown
Contributor
✅ Action performed

Review finished.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 4

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (1)
kits/isolate/apps/lib/runtime/daytona.ts (1)

344-436: 📐 Maintainability & Code Quality | 🟡 Minor | ⚡ Quick win

Biome's noUnsafeFinally is still tripping on lines 433 and 711 — the restructure didn't actually clear it.

Both blocks now correctly preserve the primary error/log the cleanup failure per the earlier review's intent, but Biome flags any throw inside a finally, conditional or not (confirmed: its rule engine doesn't reason about the surrounding if). The static analysis hints for this diff still show [error] 433-433 and [error] 711-711 for "Unsafe usage of 'throw'", so this will keep failing lint even though the runtime semantics are sound now.

Move the "throw only if the body succeeded" decision outside the finally block entirely to satisfy both the linter and the original request.

🧹 Move the conditional throw out of `finally`
-    } catch (error) {
-      probeError = error;
-      throw error;
-    } finally {
+    } catch (error) {
+      probeError = error;
+    } finally {
       let cleanupError: unknown;
       let cleaned = false;
       for (let attempt = 0; attempt < 2; attempt += 1) {
         ...
       }
-      if (!cleaned) {
-        if (probeError) {
-          console.error("Isolate probe cleanup failed after probe failure", cleanupError);
-        } else {
-          throw cleanupError;
-        }
-      }
+      if (!cleaned && probeError) {
+        console.error("Isolate probe cleanup failed after probe failure", cleanupError);
+      }
+      this.pendingProbeCleanupError = !cleaned && !probeError ? cleanupError : undefined;
     }
+    if (probeError) throw probeError;
+    if (this.pendingProbeCleanupError) throw this.pendingProbeCleanupError;

Apply the analogous restructuring to resetWorkspace's try/catch/finally around line 675-715.

Also applies to: 675-715

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@kits/isolate/apps/lib/runtime/daytona.ts` around lines 344 - 436, Restructure
the probe cleanup flow so the finally block performs cleanup and records
cleanupError but contains no throw; after the try/finally completes,
conditionally throw cleanupError only when no probeError occurred, while
preserving primary probe errors and cleanup logging. Apply the same pattern to
resetWorkspace’s try/catch/finally cleanup flow, ensuring both paths satisfy
noUnsafeFinally without changing error precedence.

Source: Linters/SAST tools

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@kits/isolate/apps/lib/investigate.ts`:
- Around line 63-82: Update assertRequiredLocalServiceStarted to detect
arbitrary repository service-script names instead of only the literal service,
server, and api keys, so local-service commands require the corresponding
package-manager run command. Preserve the existing local-URL and rejection
behavior, and escape the captured serviceScript before interpolating it into new
RegExp because repositoryContext is untrusted.

In `@kits/isolate/apps/lib/runtime/github.ts`:
- Around line 57-60: Update the request timeout handling around AbortSignal.any
to avoid relying on the affected runtime behavior: retain a strong reference to
the composed timeout signal for the request lifetime, or use a runtime-supported
equivalent that cannot be garbage-collected before firing. Preserve
options.signal composition and the 10-second timeout, and verify the deployed
Node.js runtime includes the required GC fix.

In `@kits/isolate/apps/package.json`:
- Around line 15-17: Update the dependency declarations in package.json to use
compatible Next.js 14–15 and React 18 releases, including matching react-dom and
both `@types/react` packages. Then regenerate the lockfile so it reflects the
selected versions and resolved dependency graph.

In `@kits/isolate/prompts/isolate-reproduction-system.md`:
- Line 8: Update the TUI unsaved-exit evidence contract to replace the
unsupported “bun --cwd” wording with the allowed structured workspace form “bun
run --cwd <relative-package-directory> <script>”. Preserve the existing
requirements for working-directory-relative artifact paths and all runtime-owned
fixture, input, assertion, repeat, and cleanup behavior.

---

Outside diff comments:
In `@kits/isolate/apps/lib/runtime/daytona.ts`:
- Around line 344-436: Restructure the probe cleanup flow so the finally block
performs cleanup and records cleanupError but contains no throw; after the
try/finally completes, conditionally throw cleanupError only when no probeError
occurred, while preserving primary probe errors and cleanup logging. Apply the
same pattern to resetWorkspace’s try/catch/finally cleanup flow, ensuring both
paths satisfy noUnsafeFinally without changing error precedence.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Repository UI (base), Organization UI (inherited)

Review profile: ASSERTIVE

Plan: Pro Plus

Run ID: 39f9c593-c9b1-4fa4-9812-1fbd0bec986d

📥 Commits

Reviewing files that changed from the base of the PR and between f36ec09 and 449590a.

📒 Files selected for processing (24)
  • kits/isolate/README.md
  • kits/isolate/agent.md
  • kits/isolate/apps/lib/investigate.ts
  • kits/isolate/apps/lib/investigation-request.ts
  • kits/isolate/apps/lib/lamatic-planner.ts
  • kits/isolate/apps/lib/runtime/claim.ts
  • kits/isolate/apps/lib/runtime/daytona.ts
  • kits/isolate/apps/lib/runtime/evidence.ts
  • kits/isolate/apps/lib/runtime/github.ts
  • kits/isolate/apps/lib/runtime/mcp.ts
  • kits/isolate/apps/lib/runtime/policy.ts
  • kits/isolate/apps/package.json
  • kits/isolate/apps/tests/claim.test.ts
  • kits/isolate/apps/tests/concurrency.test.ts
  • kits/isolate/apps/tests/daytona.test.ts
  • kits/isolate/apps/tests/evidence.test.ts
  • kits/isolate/apps/tests/investigate.test.ts
  • kits/isolate/apps/tests/investigation-report.test.ts
  • kits/isolate/apps/tests/investigation-request.test.ts
  • kits/isolate/apps/tests/lamatic-planner.test.ts
  • kits/isolate/apps/tests/mcp.test.ts
  • kits/isolate/apps/tsconfig.json
  • kits/isolate/lamatic.config.ts
  • kits/isolate/prompts/isolate-reproduction-system.md

Comment thread kits/isolate/apps/lib/investigate.ts Outdated
Comment thread kits/isolate/apps/lib/runtime/github.ts Outdated
Comment thread kits/isolate/apps/package.json Outdated
Comment thread kits/isolate/prompts/isolate-reproduction-system.md Outdated
@Dhruv2mars

Copy link
Copy Markdown
Author

@coderabbitai review

@coderabbitai

coderabbitai Bot commented Jul 30, 2026

Copy link
Copy Markdown
Contributor
✅ Action performed

Review finished.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (1)
kits/isolate/apps/lib/investigate.ts (1)

161-164: 🎯 Functional Correctness | 🟡 Minor | ⚡ Quick win

Mission update: the "bun --cwd" briefing never got the memo.

This runtime-injected evidenceGuidance string still tells the planner "when command uses bun --cwd, adjust the path" — the exact wording already corrected to bun run --cwd in kits/isolate/prompts/isolate-reproduction-system.md. Left as-is, a TUI-mode plan following this instruction literally can emit the unsupported bun --cwd runner form, which the command policy rejects, costing a plan-repair retry for the whole tui_unsaved_exit evidence path.

🕵️ Sync the wording with the fixed prompt file
-Resolve every build-artifact path from the effective working directory of command; when command uses bun --cwd, adjust the path for that directory.
+Resolve every build-artifact path from the effective working directory of command; when command uses bun run --cwd, adjust the path for that directory.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@kits/isolate/apps/lib/investigate.ts` around lines 161 - 164, Update the
`evidenceGuidance` text for `assertion?.kind === "tui_unsaved_exit"` to refer to
the supported `bun run --cwd` form instead of `bun --cwd`, while preserving the
existing path-adjustment instruction and all other guidance.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Outside diff comments:
In `@kits/isolate/apps/lib/investigate.ts`:
- Around line 161-164: Update the `evidenceGuidance` text for `assertion?.kind
=== "tui_unsaved_exit"` to refer to the supported `bun run --cwd` form instead
of `bun --cwd`, while preserving the existing path-adjustment instruction and
all other guidance.

ℹ️ Review info
⚙️ Run configuration

Configuration used: Repository UI (base), Organization UI (inherited)

Review profile: ASSERTIVE

Plan: Pro Plus

Run ID: 783ba816-1b42-45f9-8a7e-4ace6c7baa69

📥 Commits

Reviewing files that changed from the base of the PR and between 449590a and 0d67c1d.

⛔ Files ignored due to path filters (1)
  • kits/isolate/apps/bun.lock is excluded by !**/*.lock
📒 Files selected for processing (13)
  • kits/isolate/apps/lib/investigate.ts
  • kits/isolate/apps/lib/runtime/daytona.ts
  • kits/isolate/apps/lib/runtime/github.ts
  • kits/isolate/apps/lib/runtime/investigation-report.ts
  • kits/isolate/apps/next-env.d.ts
  • kits/isolate/apps/package.json
  • kits/isolate/apps/styles.d.ts
  • kits/isolate/apps/tests/investigate.test.ts
  • kits/isolate/apps/tests/investigation-report.test.ts
  • kits/isolate/apps/tests/lamatic-planner.test.ts
  • kits/isolate/apps/tests/plan.test.ts
  • kits/isolate/apps/tsconfig.json
  • kits/isolate/prompts/isolate-reproduction-system.md

@Dhruv2mars

Dhruv2mars commented Jul 30, 2026

Copy link
Copy Markdown
Author

@akshatvirmani Done: the PR is ready for final review. All CodeRabbit findings are addressed and resolved, CodeRabbit approved the exact current head, repository validation and the full local test/typecheck/build gate pass, and the production demo is deployed and Ready.

@github-actions

Copy link
Copy Markdown
Contributor

Hi @Dhruv2mars! 👋

Before this PR can be reviewed by maintainers, please resolve all comments and requested changes from the CodeRabbit automated review.

Steps to follow:

  1. Read through all CodeRabbit comments carefully
  2. Address each issue raised (or reply explaining why you disagree)
  3. Push your fixes as new commits
  4. Once all issues are resolved, comment here so we can re-review

This helps keep the review process efficient for everyone. Thank you! 🙏

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (1)
kits/isolate/apps/lib/investigate.ts (1)

280-289: 🎯 Functional Correctness | 🟡 Minor | ⚡ Quick win

Keep the Agent hypothesis field consistent with the remaining investigation paths.

This exploratory evidence review assigns report.summary to hypothesis, while certified and TUI paths return the planner hypothesis and the UI labels result.hypothesis as the preliminary Lamatic hypothesis. Use the planner hypothesis if this field must continue representing the agent’s planner hypothesis; otherwise adjust the field name/contract and update the UI to prevent calling the post-runtime summary an agent hypothesis.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@kits/isolate/apps/lib/investigate.ts` around lines 280 - 289, Update the
exploratory evidence-review return object in the investigate flow to use the
planner’s hypothesis for the `hypothesis` field, matching the certified and TUI
paths, instead of `report.summary`. Preserve the existing report summary in its
appropriate report field, and keep the `result.hypothesis` UI label consistent
with the value’s planner-hypothesis contract.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Outside diff comments:
In `@kits/isolate/apps/lib/investigate.ts`:
- Around line 280-289: Update the exploratory evidence-review return object in
the investigate flow to use the planner’s hypothesis for the `hypothesis` field,
matching the certified and TUI paths, instead of `report.summary`. Preserve the
existing report summary in its appropriate report field, and keep the
`result.hypothesis` UI label consistent with the value’s planner-hypothesis
contract.

ℹ️ Review info
⚙️ Run configuration

Configuration used: Repository UI (base), Organization UI (inherited)

Review profile: ASSERTIVE

Plan: Pro Plus

Run ID: 3aa13f47-e669-4460-ba34-8fcf3a224640

📥 Commits

Reviewing files that changed from the base of the PR and between 0d67c1d and f0967d7.

📒 Files selected for processing (2)
  • kits/isolate/apps/lib/investigate.ts
  • kits/isolate/apps/tests/investigate.test.ts

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants